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Perceptual  Issues  in  the  Use  of  Head-Mounted  Visual 
Displays 


Robert  Patterson,  Washington  State  University,  Pullman,  Washington,  and  Marc  D. 
Winterbottom  and  Byron  J.  Pierce,  Air  Force  Research  Laboratory,  Mesa,  Arizona 

Objective:  We  provide  a  review  and  analysis  of  much  of  the  published  literature  on 
visual  perception  issues  that  impact  the  design  and  use  of  head-mounted  displays 
(HMDs).  Background:  Unlike  the  previous  literature  on  HMDs,  this  review  draws 
heavily  from  the  basic  vision  literature  in  order  to  help  provide  insight  for  future  design 
solutions  for  HMDs.  Method:  Included  in  this  review  are  articles  and  books  found 
cited  in  other  works  as  well  as  articles  and  books  obtained  from  an  Internet  search. 

Results:  Issues  discussed  include  the  effect  of  brightness  and  contrast  on  depth  of  held, 
dark  focus,  dark  vergence,  and  perceptual  constancy;  the  effect  of  accommodation- 
vergence  synergy  on  perceptual  constancy,  eyestrain,  and  discomfort;  the  relationship 
of  held  of  view  to  the  functioning  of  different  visual  pathways  and  the  types  of  visual- 
motor  tasks  mediated  by  them;  the  relationship  of  binocular  input  to  visual  suppres¬ 
sion;  and  the  importance  of  head  movements,  head  tracking,  and  display  update  lag. 
Conclusion:  This  paper  offers  a  set  of  recommendations  for  the  design  and  use  of 
HMDs.  Application:  Consideration  of  the  basic  vision  literature  will  provide  insight 
for  future  design  solutions  for  HMDs. 

INTRODUCTION 

One  important  technological  advance  in  recent 
years  has  been  the  development  of  wearable  head- 
mounted  visual  displays  (HMDs)  for  military  and 
commercial  applications  (e.g.,  Frey  &  Page,  2001 ; 

Licina,  Rash,  Mora,  &  Ledford,  1999).  An  HMD 
presents  symbolic  or  pictorial  information  to  the 
eyes  of  a  user  by  way  of  one  or  two  miniature  visu¬ 
al  displays,  such  as  an  aiming  reticle  or  full-color 
imagery  (Havig,  Grigsby,  Heft,  LaCreta,  &  Post, 

200 1 ),  mounted  on  the  head  via  a  helmet  or  other 
kind  of  arrangement.  If  implemented  properly, 

HMDs  can  offer  advantages  over  traditional  dis¬ 
plays,  such  as  increased  situational  awareness  and 
ease  of  mobility  (Velger,  1998).  Thus,  HMDs  are 
increasingly  being  considered  for  use  in  a  wide  va¬ 
riety  of  applications,  including  surgery  and  other 
medical  applications,  remote  vehicle  operation, 
flight  simulation,  and  automobile  racing  (Melzer 
&  Moffitt,  1997;  Velger,  1998). 

Despite  the  potential  advantages  of  HMDs, 
their  development  through  the  years  has  not  been 


uniform  (Ruston,  Mon- Williams,  &  Wann,  1994; 
Sheedy  &  Bergstrom,  2002).  Although  Peli  (1998) 
noted  that  visual  effects  during  the  use  of  an  HMD 
were  similar  to  those  of  a  desktop  display,  Keller 
and  Colucci  (1998)  stated  that  HMDs  have  often 
disappointed  real-world  users.  Although  HMDs 
have  been  used  successfully  in  certain  applica¬ 
tions,  their  use  in  other  applications  has  been 
troublesome  (Velger,  1998).  This  is  attributable,  in 
part,  to  the  fact  that  HMDs  can  create  significant 
perceptual  problems  not  found  with  traditional 
displays  (Melzer  &  Moffitt,  1997).  For  example, 
Wenzel,  Castillo,  and  Baker  (2002)  found  that  air¬ 
craft  maintenance  workers  who  used  an  HMD  for 
training  purposes  reported  problems  such  as  eye- 
strain,  headache,  nausea,  and  dizziness.  Further¬ 
more,  Morphew,  Shively,  and  Casey  (2004)  found 
that  self-reports  of  nausea,  disorientation,  and  ocu¬ 
lomotor  strain  were  greater  with  an  HMD  as  com¬ 
pared  with  a  standard  computer  monitor  for  an 
unmanned  aerial  vehicle  control  task.  Moreover, 
Hakkinen  (2004)  reported  similar  problems  when 
a  monocular  HMD  was  used  during  performance 
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of  a  text-editing  task.  Finally,  Kooi  (1997)  report¬ 
ed  significantly  greater  eyestrain  with  the  use  of 
HMDs  as  compared  with  a  standard  computer 
monitor.  Simulator  sickness  has  also  been  report¬ 
ed  with  the  use  of  HMDs  and  can  be  caused  by  a 
number  of  factors  (Draper,  Viirre,  Furness,  & 
Gawron,  2001 ;  Draper,  Viirre,  Furness,  &  Parker, 
1997;  Ehrlich,  1997). 

These  problems  and  others  arise  because  HMDs 
present  an  unnatural  viewing  situation.  First  con¬ 
sider  the  situation  in  which  an  observer  bino- 
cularly  views  the  world  under  natural  viewing 
conditions.  In  this  case,  the  image  projection  sys¬ 
tem  is  linked  to  the  environment  insofar  as  light 
is  reflected  off  surfaces  and  objects  and  projected 
to  the  user’s  two  eyes.  The  views  delivered  to  the 
two  eyes  would  be  similar  and  carry  binocular  dis¬ 
parity  (also  called  binocular  parallax)  informa¬ 
tion.  Moreover,  head  movements  would  likely 
alter  the  pattern  of  retinal  stimulation,  as  would 
eye  movements.  A  similar  situation  would  occur 
when  an  observer  views  a  typical  computer  dis¬ 
play:  The  location  of  the  display  is  fixed  in  the 
environment,  the  same  or  similar  imagery  is  pro¬ 
jected  to  the  user’s  two  eyes,  and  head  and  eye 
movements  would  likely  alter  the  pattern  of  reti¬ 
nal  stimulation. 

Now  consider  HMDs.  They  can  create  signif¬ 
icant  perceptual  problems  because  the  image  pro¬ 
jection  system  is  linked  to  the  user’s  head,  which 
is  unnatural.  The  views  delivered  to  the  two  eyes 
can  be  very  different  in  the  case  of  monocular 
HMDs,  in  which  one  eye  sees  the  HMD  and  the 
other  eye  views  a  real-world  scene .  The  two  eyes  ’ 
views  can  be  different  in  the  case  of  binocular 
HMDs  as  well,  because  the  two  eyes  view  slight¬ 
ly  different  imagery  if  there  exists  significant  opti¬ 
cal  misalignment  or  image  distortion  between  the 
two  eyes’  views.  Moreover,  in  the  case  of  HMDs, 
head  movements  would  not  alter  the  pattern  of  ret¬ 
inal  stimulation  (but  eye  movements  could)  un¬ 
less  a  computer  were  made  to  shift  and  update  the 
imagery  shown  on  the  HMD  each  time  the  user’s 
head  moved. 

Our  goal  in  writing  this  review  paper  was  sim¬ 
ply  to  research  perceptual  issues  relevant  to  the  use 
of  HMDs  for  flight  simulation  and  training  appli¬ 
cations.  We  found  some  excellent  books  describ¬ 
ing  HMD  engineering  design  and  technology 
issues  (e.g.,  Melzer  &  Moffitt,  1997;  Velger,  1998) 
and  a  plethora  of  research  on  individual  issues 


related  to  HMDs  but  not  necessarily  tied  exten¬ 
sively  to  the  basic  visual  research  literature. 

It  is  our  belief  that  new  insights  for  potential 
design  solutions  for  HMDs  are  to  be  gained  by  a 
thorough  understanding  of  basic  visual  function¬ 
ing.  Even  when  a  potential  solution  for  a  given 
problem  is  not  immediately  obvious,  knowledge 
about  basic  visual  functioning  should  have  the  po¬ 
tential  of  informing  the  development  of  a  solution 
at  a  later  time.  Therefore,  we  have  drawn  upon  the 
basic  vision  research  literature  in  an  effort  to  un¬ 
derstand  perceptual  issues  that  may  hinder  the  use 
of  HMDs  in  flight  simulation  and  training  appli¬ 
cations.  Consequently,  our  review  covers  basic 
perceptual  issues  that  are  not  often  extensively 
covered  in  other  sources,  such  as  perceptual  con¬ 
stancy,  accommodation- vergence  synergy,  visual 
pathway  functioning,  and  visual  suppression.  Thus, 
this  review  is  novel  in  the  extent  to  which  the  basic 
vision  literature  is  cited  and  discussed  when  con¬ 
sidering  potential  design  solutions  for  HMDs. 
Approximately  one  half  of  the  articles  and  books 
covered  in  this  review  derive  from  the  basic  vision 
literature. 

The  following  section  covers  the  basic  compo¬ 
nents  of  a  generic  HMD  system.  That  section  is 
followed  by  sections  on  five  key  perceptual  issues 
associated  with  the  use  of  HMDs.  Finally,  the  last 
section  concludes  with  a  summary  of  recommen¬ 
dations  for  the  use  of  HMDs. 

A  GENERIC  HMD  SYSTEM 

Many  HMD  systems  possess  common  compo¬ 
nents:  a  type  of  information  to  be  displayed,  a  dis¬ 
play  device  attached  to  the  head,  relay  optics,  a 
combiner  if  the  imagery  is  seen  as  superimposed 
on  another  display  screen  or  on  the  outside  world, 
and  a  head-tracking  system  (Velger,  1998).  We 
briefly  discuss  the  components  for  a  generic  HMD 
system. 

Type  of  Information 

Different  types  of  information  can  be  displayed 
on  an  HMD.  These  types  are  symbology,  text  and 
graphics,  and  real-time  imagery  with  movement. 
The  type  of  information  displayed  will  depend  up¬ 
on  the  particular  application  under  consideration. 

In  military  applications  information  involving 
vehicle  status,  the  aiming  of  weapons,  sensors,  or  a 
target’s  location  transmitted  from  other  individuals 
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may  be  displayed.  In  commercial  applications,  in¬ 
formation  involving  maintenance  and  repair  may 
be  displayed.  In  entertainment  applications,  infor¬ 
mation  involving  gaming,  movies,  or  personal 
digital  assistants  (PDAs)  may  be  displayed  (Hak- 
kinen,  2004). 

Display  Devices 

Many  types  of  visual  displays  could  potential¬ 
ly  be  used  in  HMD  applications.  Miniature  mono¬ 
chrome  CRT  displays  have  generally  been  used 
with  HMDs  in  the  past  because  of  their  high  lu¬ 
minance  output,  low  cost,  durability,  and  desirable 
temporal  characteristics.  Significant  disadvan¬ 
tages  of  CRT  displays  include  their  relatively  high 
weight,  large  size,  and  high  power  requirements 
(Velger,  1998). 

Liquid  crystal  on  silicon  (LCoS)  displays  are 
relatively  fast  and  have  good  light  output,  low  po¬ 
wer  consumption,  low  weight,  small  size,  and 
relatively  high  resolution.  One  disadvantage  of 
current  LCoS  displays,  however,  is  that  the  LCoS 
pixels  are  illuminated  for  an  entire  video  frame, 
which  may  cause  motion  artifacts  such  as  blurring 
or  double  images  (Lindholm,  Pierce,  &  Scharine, 
2001 ;  Winterbottom,  Geri,  &  Pierce,  2004).  This 
is  not  a  major  concern  in  applications  in  which 
only  static  text  or  graphics  is  viewed.  For  appli¬ 
cations  involving  moving  imagery,  however,  the 
imagery  will  appear  to  lose  definition.  Anew  type 
of  LCoS  display,  called  a  ferro-electric  LCoS 
(FLCoS)  display,  could  provide  a  solution  to  the 
pixel  illumination  issue  (Velger,  1998). 

Virtual  retinal  displays  (VRDs)  present  images 
to  the  visual  system  by  scanning  the  images  direct¬ 
ly  onto  the  retina  using  a  diode  laser  light  source 
and  thus  eliminating  the  display  screen.  The  poten¬ 
tial  advantages  of  such  a  system  are  increased 
resolution,  contrast,  brightness,  and  color  quality, 
and  decreased  weight.  The  bright  high-resolution 
image  source  can  be  located  off-helmet  and  its 
image  presented  to  the  helmet  and  pilot  via  fiber¬ 
optic  image  guides  (Velger,  1998,  pp.  109).  One 
disadvantage  of  this  display  device  for  HMD  ap¬ 
plications  is  the  small  eyebox.  Slippage  of  the  hel¬ 
met,  or  large  amplitude  eye  movements,  may  cause 
portions  of  the  image  to  disappear  from  view. 

Digital  light-processing  (DLP)  displays,  or  dig¬ 
ital  micro  mirror  displays,  operate  using  tiny  ad¬ 
dressable  micromirrors  that  pivot  between  two 
orientations  to  reflect  light  out  through  the  lens  to 


the  observer  or  reflect  it  into  an  absorptive  mate¬ 
rial.  Because  the  micromirrors  can  be  switched  at 
a  high  speed,  such  displays  have  the  potential  for 
presenting  good  quality  motion.  However,  this 
potential  may  be  undercut  by  the  fact  that  current 
DLP  displays,  like  the  LCoS  displays,  are  designed 
to  illuminate  each  pixel  for  an  entire  video  frame. 

Generally,  the  display  device  is  driven  by  one 
or  more  imaging  systems,  such  as  a  digital  cam¬ 
era  or  sensor  attached  to  a  moving  vehicle,  or  an 
image  generator  (e.g.,  a  computer  providing  video 
output).  HMDs  used  for  entertainment  purposes 
may  be  driven  by  a  DVD  player,  VCR,  computers 
equipped  with  video  output,  or  a  PDA. 

Optics 

HMDs  use  several  types  of  optical  elements, 
which  serve  to  magnify  and  deliver  images  to  the 
eye  or  eyes  of  the  user  (e.g.,  Rash,  Kalich,  &  van 
de  Pol,  2002).  These  include  diffraction  optics 
and  holographic  optical  elements  (Velger,  1998, 
pp.  115-129;  Wood,  1992).  The  different  types 
of  optical  designs  result  in  varying  trade-offs  in 
weight,  image  quality,  field  of  view,  and  light 
transmittance. 

Briefly,  the  optical  elements  of  an  HMD,  to¬ 
gether  with  associated  hardware,  can  be  arranged 
to  create  one  of  three  types  of  HMD  (Davis,  1997): 
a  monocular  HMD,  wherein  imagery  is  present¬ 
ed  to  one  eye  of  the  user  while  the  other  eye  views 
the  real  world  (the  eye  that  receives  the  HMD 
imagery  may  also  view  the  real  world);  a  biocu¬ 
lar  HMD,  wherein  the  same  imagery  is  presented 
to  both  eyes  of  the  user  (P.  J.  Rogers  &  Freeman, 
1992);  or  a  binocular  HMD,  wherein  imagery  with 
binocular  disparity  is  presented  to  the  two  eyes  of 
the  user  (Klymenko,  Verona,  Martin,  &  Beasley, 
1994a,  1994b). 

According  to  Tsou  and  Shenker  (2000),  a  given 
HMD  system  can  be  closed  or  occluded  in  that  a 
direct  view  of  the  outside  world  is  not  possible 
because  it  is  blocked  or  the  lighting  is  too  dark  to 
see;  the  user  sees  only  the  HMD  imagery.  Alter¬ 
natively,  an  HMD  system  can  be  open  or  semi¬ 
transparent,  allowing  a  direct  view  of  the  outside 
world,  and  the  user  views  the  HMD  imagery  as 
superimposed  upon  the  outside  world. 

Head  Tracking 

Some  HMD  systems  employ  conformal  image¬ 
ry.  Conformal  imagery  refers  to  world- stabilized 
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imagery  that  conforms  to  the  outside  world  and  is 
aligned  with  local  geography  (Velger,  1998).  To 
create  world-stabilized  imagery,  the  imagery  has 
to  change  with  head  position  in  order  to  simulate 
the  shift  in  scene  that  would  naturally  occur  with 
changes  in  head  position  when  viewing  the  real 
world.  To  do  so,  head  position  has  to  be  measured 
and  tracked.  Conformal  imagery  is  difficult  to  im¬ 
plement  because  it  is  difficult  to  accurately  mea¬ 
sure  head  position  and  quickly  update  the  imagery 
within  a  brief  period  (Velger,  1998). 

Head  trackers  typically  measure  six  degrees 
of  freedom  of  head  position:  three  angular  orien¬ 
tations  of  roll,  pitch,  and  yaw  and  three  linear 
positions  of  x,  y,  and  z  (Velger,  1998,  pp.  78, 143). 
Measurement  of  three  degrees  of  freedom  is  suffi¬ 
cient  for  applications  that  do  not  require  exact 
positioning  of  imagery  relative  to  head  movement. 
However,  this  type  of  system  may  not  measure 
head  roll  -  cocking  the  head  slightly  to  the  left  or 
right,  for  example. 

Head  trackers  can  be  electromagnetic,  electro- 
optical,  or  ultrasonic.  There  are  also  inertial  head 
trackers  with  mounted  miniature  gyroscopes  that 
can  measure  high  rates  of  head  rotation  (Meyer, 
Applewhite  &  Biocca,  1992;  Velger,  1998).  Each 
type  of  head  tracker  presents  problems.  For  exam¬ 
ple,  the  electromagnetic  trackers  are  affected  by 
nearby  metal  objects  and  electromagnetic  radia¬ 
tion  (e.g.,  Ferrin,  1991),  the  electro-optical  track¬ 
ers  require  a  direct  line  of  sight  and  large  held  of 
view,  and  the  ultrasonic  trackers  can  be  contam¬ 
inated  by  external  acoustic  noise  (Velger,  1998, 
pp.  80,  165). 

Anderson,  Vrana,  Riegler,  and  Martin  (2002) 
evaluated  several  types  of  head-tracking  systems 
for  use  with  night  vision  training.  Inertial  head 
trackers  were  recommended  for  enclosed  areas 
where  the  placement  of  additional  sensors  is  not 
possible.  Optical  head  trackers  were  recommended 
for  conditions  in  which  no  instruments,  switches, 
or  visual  displays  are  located  above  the  observer’s 
head.  Magnetic  trackers  were  recommended  for 
small  operating  spaces.  (Note  that  magnetic  and 
optical  head  trackers  may  have  an  advantage  over 
inertial  trackers  because  they  provide  for  the  mea¬ 
surement  of  heading,  pitch,  and  roll  as  well  as  of 
x,  y,  and  z  coordinates.) 

According  to  Velger  (1998,  pp.  144-145),  the 
following  ranges  of  head  movements  should  be 
measurable  with  any  given  head  tracking  system: 


1 80°  for  angular  azimuth,  130°  for  elevation,  and 
120°  for  roll,  with  an  accuracy  of  about  1  to  2  mil- 
liradians  (mrad)  on  boresight  and  about  2  to  6  mrad 
at  a  10°  eccentricity,  and  linear  displacements  of 
450  mm  in  the  vertical  axis,  400  mm  in  the  hori¬ 
zontal  axis,  and  540  mm  in  the  fore-aft  axis. 

In  summary,  there  are  many  types  of  HMDs, 
depending  upon  the  type  of  information  displayed, 
the  type  of  display  employed,  the  type  of  imaging 
system  employed,  the  type  of  optical  arrangement 
used,  and  the  type  of  head  tracker  used.  The  HMD 
system  can  be  monocular,  biocular,  or  binocular, 
and  the  optical  arrangement  can  be  either  closed 
or  semitransparent. 

Now  that  we  have  covered  the  basic  compo¬ 
nents  of  a  generic  HMD  system,  we  turn  to  the  dis¬ 
cussion  of  several  key  perceptual  issues  that  are 
relevant  to  the  design  and  use  of  HMDs. 

BRIGHTNESS  AND  CONTRAST 

Brightness  and  contrast  are  important  consid¬ 
erations  when  using  HMDs  insofar  as  the  imagery 
should  be  clearly  visible  when  one  is  wearing  the 
HMD.  Interestingly,  failure  to  provide  imagery 
that  is  bright  and  of  high  contrast  has  perceptual 
consequences  other  than  just  poor  visibility.  These 
perceptual  consequences  will  be  discussed. 

Depth  of  Field 

When  an  HMD  is  worn  while  one  is  viewing  a 
real  out-the-window  (OTW)  scene,  such  as  with 
helmet-mounted  sights,  both  the  HMD  imagery 
and  the  OTW  view  must  be  clearly  visible  at  the 
same  time.  For  this  to  occur,  the  HMD  imagery 
and  any  objects  in  the  OTW  view  must  be  within 
the  user’s  depth  of  held.  Depth  of  field  refers  to  the 
range  of  distances  in  object  space  within  which 
stimuli  appear  in  sharp  focus.  One  factor  that  de¬ 
termines  the  user’s  depth  of  held  is  the  focal  dis¬ 
tance  at  which  the  HMD  is  set.  Establishing  that 
the  HMD  imagery  and  the  objects  in  the  OTW 
view  are  all  within  the  user’s  depth  of  held  should 
not  be  a  signihcant  problem  because  the  HMD  op¬ 
tics  are  typically  collimated  for  this  application. 
That  is,  the  focal  distance  of  the  HMD  is  set  to  near 
optical  inhnity  so  that  both  the  HMD  imagery  and 
the  OTW  scene  are  in  focus  simultaneously.  How¬ 
ever,  when  one  wears  an  HMD  while  viewing 
another  synthetic  vision  display,  such  as  the  visu¬ 
al  display  of  a  flight  simulator,  both  HMD  imagery 
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and  the  simulator  imagery  must  be  clearly  visible 
at  the  same  time.  For  this  to  occur,  both  the  HMD 
and  simulator  imagery  must  be  within  the  user’s 
depth  of  held. 

For  the  dome  type  of  simulator  displays,  the 
focal  distance  of  the  HMD  would  be  set  to  a  value 
that  equals  the  radius  of  the  dome.  However,  for 
multifaceted  simulator  displays,  when  the  corner 
of  the  background  display  is  viewed  from  the  ob¬ 
server’s  position  (i.e.,  off-axis  viewing),  the  view¬ 
ing  distance  would  be  greater  than  when  the  display 
is  viewed  in  the  straight-ahead  position.  This  rais¬ 
es  the  question  of  whether  the  focal  distance  of 
the  HMD  should  be  set  to  a  value  that  matches  the 
distance  of  the  straight-ahead  view,  to  a  value  that 
matches  the  distance  of  the  off-axis  view,  or  to  an 
intermediate  value. 

This  issue  could  be  tested  by  using  an  acuity 
measure  and  presenting  targets  on  an  HMD  for  a 
duration  shorter  than  the  latency  of  the  accom¬ 
modative  response,  which  can  range  from  about 
370  ms  to  1  s  or  more  (Campbell  &  Westheimer, 
1960).  This  approach  could  be  used  to  measure 
depth  of  held  for  a  given  application  and  deter¬ 
mine  whether  the  focal  distances  of  two  different 
displays  are  within  the  user’s  depth  of  held. 

Brightness  and  contrast  of  the  HMD  symbolo¬ 
gy  can  affect  the  user’s  depth  of  held,  as  can  pupil 
size  and  level  of  resolution.  Generally,  increases 
in  luminance  level  produce  a  smaller  pupil,  which 
leads  to  a  greater  depth  of  held.  For  example,  Ogle 
and  Schwartz  (1959)  report  that  for  each  milli¬ 
meter  of  increase  in  pupil  size  (ranging  from  2.5 
to  8.0  mm),  depth  of  focus  decreased  by  0.12 
diopters.  ( Depth  of  focus  refers  to  the  range  of 
distances  in  image  space  within  which  an  image 
appears  in  sharp  focus.)  Conversely,  a  greater  de¬ 
gree  of  resolution  will  lead  to  a  smaller  depth  of 
held  (Campbell,  1957;  Ogle  &  Schwartz,  1959). 
Ogle  and  Schwartz  (1959)  showed  that  for  each 
step  of  increase  in  target  resolution,  the  total  depth 
of  focus  decreased  by  0.35  diopters.  Thus,  one  can¬ 
not  easily  specify  a  given  difference  in  focal  dis¬ 
tances  that  individuals  can  tolerate;  a  number  of 
factors  enter  into  the  determination  of  the  depth 
of  held.  Thus,  it  is  important  to  take  into  account 
all  relevant  factors  when  designing  an  HMD 
system. 

Dark  Focus  and  Dark  Vergence 

When  considering  the  focal  distance  of  the 


HMD,  one  needs  to  take  into  account  the  dark  fo¬ 
cus  of  human  observers.  Dark  focus  refers  to  the 
tendency  of  accommodation  to  drift  toward  a  rest¬ 
ing  distance  of  approximately  1  m  from  the  observ¬ 
er,  which  occurs  more  under  degraded  stimulus 
conditions  (Hennessy,  Iida,  Shina  &  Leibowitz, 
1976;  Leibowitz  &  Owens,  1975a,  1975b).  Forex- 
ample,  Leibowitz  and  Owens  (1975b)  measured 
accommodation  while  a  building  was  viewed  at  a 
distance  of 200  m  under  bright  daylight  conditions 
and  when  the  same  scene  was  viewed  through 
neutral  density  hlters,  which  lowered  luminance 
level.  These  authors  found  that  whereas  the  ac¬ 
commodative  response  was  driven  mainly  by  the 
stimulus  (building)  under  bright  viewing  con¬ 
ditions,  the  accommodative  response  became  a 
compromise  between  the  response  driven  by  the 
stimulus  and  the  dark  focus  value  when  lumi¬ 
nance  level  was  reduced  by  1 .95  log  units.  (On  the 
assumption  that  the  luminance  level  of  a  bright 
outdoor  scene  can  be  on  the  order  of  34,262.6 
cd/m2  [Velger,  1998,  p.  90],  this  would  make  the 
luminance  level  roughly  342.6  cd/m2.)  The  ac¬ 
commodative  response  was  close  to  being  the  dark 
focus  value  when  luminance  level  was  reduced 
by  4.2  log  units  (an  estimated  luminance  level  of 
roughly  3.4  cd/m2).  However,  there  are  large  in¬ 
dividual  differences  in  the  actual  dark  focus  value 
when  measured  across  individuals  (Leibowitz  & 
Owens,  1975a),  and  it  likely  would  not  be  possi¬ 
ble  to  predict  in  advance  what  any  given  individ¬ 
ual’s  dark  focus  value  would  be. 

In  general,  HMD  focal  distances  that  are  in¬ 
creasingly  farther  from  an  individual’s  dark  focus 
value  will  tend  to  produce  errors  of  accommoda¬ 
tion  and  poor  visual  resolution,  especially  under 
conditions  of  low  illumination  (e.g.,  below  about 
6.9  cd/m2;  see  Johnson,  1976).  Owens  (1979) 
showed  that  the  accommodative  response  will  be 
biased  toward  the  distance  that  is  closer  to  the 
user’s  dark  focus  value;  the  stimulus  near  the  dark 
focus  value  will  require  less  accommodative  effort 
(for  a  contrasting  view,  see  Gleason  &  Kenyon, 
1997).  This  may  have  significance  for  situations 
in  which  two  stimuli  with  different  focal  distances 
are  optically  superimposed,  such  as  when  an  ob¬ 
server  views  an  OTW  scene  through  a  semi¬ 
transparent  HMD  with  a  focal  distance  that  is  not 
close  to  optical  infinity. 

There  is  also  an  analogous  phenomenon  of  dark 
vergence,  which  refers  to  the  tendency  of  vergence 
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to  drift  toward  a  resting  distance  of  a  little  more 
than  1  m  from  the  observer.  In  a  study  by  Ivanoff 
(1955),  convergence  was  close  to  the  angle  corre¬ 
sponding  to  the  target  distance  for  luminance  val¬ 
ues  at  and  above  0.1  cd/m2.  (For  typical  HMD 
applications,  this  latter  value  would  represent  a 
very  low  and  unrealistic  luminance  level.)  For 
luminance  levels  below  that  value,  convergence 
approached  the  dark  vergence  angle. 

Perceptual  Constancy 

The  tendency  for  accommodation  to  drift  to¬ 
ward  dark  focus  and  vergence  to  drift  toward  dark 
vergence,  especially  under  degraded  stimulus  con¬ 
ditions,  may  cause  the  size,  depth,  or  speed  of  the 
HMD  imagery  to  be  misperceived.  Such  misper¬ 
ception  would  be  related  to  misperceived  distance 
and  the  concept  of  perceptual  constancy  (Patterson 
&  Martin,  1992),  as  we  will  explain. 

Stimuli  can  be  subdivided  into  two  categories, 
distal  and  proximal  stimuli  (Epstein,  1997).  The 
term  distal  stimuli  refers  to  the  stimuli  that  exist 
out  in  the  environment;  the  term  proximal  stimuli 
refers  to  the  stimuli  that  impinge  upon  the  eye’s 
rods  and  cones.  The  function  of  the  visual  system 
is  to  glean  information  about  distal  stimuli  from 
information  carried  by  proximal  stimuli,  a  task 
made  difficult  by  the  existence  of  changing  envi¬ 
ronmental  conditions.  For  example,  consider  the 
perception  of  size.  When  an  observer  moves  clos¬ 
er  to  an  object,  retinal  image  size  increases,  and  in 
order  for  the  observer  to  correctly  perceive  that  the 
object’s  actual  size  (distal  stimulus)  did  not  vary 
(which  is  called  size  constancy ),  the  visual  system 
likely  combines  information  about  visual  angle 
and  viewing  distance  (Foley,  Ribeiro-Filho,  & 
Da  Silva,  2004),  which  involve  proximal  stimuli. 

Similarly,  consider  the  perception  of  stereo¬ 
scopic  depth,  which  is  the  perception  of  depth 
based  upon  binocular  disparity.  When  an  observer 
moves  closer  to  a  set  of  objects  that  vary  in  depth, 
binocular  disparity  increases,  and  in  order  for  an 
observer  to  correctly  perceive  that  the  actual  depth 
relations  among  the  stimuli  (distal  stimuli)  did  not 
vary  (which  is  called  depth  constancy),  the  visual 
system  likely  combines  information  about  binoc¬ 
ular  disparity  and  viewing  distance  (Howard  & 
Rogers,  1995, 2002;  Ono  &  Comerford,  1977;  Pat¬ 
terson,  Moe,  &  Hewitt,  1992;  Ritter,  1977),  which 
involve  proximal  stimuli. 

An  analogous  phenomenon  may  occur  with 


perceived  speed.  When  an  observer  moves  closer 
to  a  laterally  moving  object,  the  speed  of  the  ob¬ 
ject’s  retinal  image  increases,  and  in  order  for  the 
observer  to  correctly  perceive  that  the  actual  speed 
of  the  object  did  not  change  (which  is  called 
speed  constancy ),  the  visual  system  may  combine 
information  about  retinal  speed  and  viewing  dis¬ 
tance.  In  this  case,  however,  the  existence  of  speed 
constancy  is  controversial.  Some  authors  (Pierce 
&  Geri,  1997;  Wallach,  1939;  Zohary  &  Sittig, 
1993)  have  found  evidence  for  speed  constancy, 
at  least  under  some  conditions,  but  other  authors 
(McKee  &  Welch,  1989)  have  not. 

In  the  three  types  of  constancies  mentioned 
previously,  size,  depth,  and  speed,  the  visual  sys¬ 
tem  is  said  to  scale  retinal  size,  binocular  dispar¬ 
ity,  or  retinal  speed,  respectively,  in  accordance 
with  viewing  distance  information,  a  process 
called  distance  scaling.  One  issue  in  the  literature 
regarding  perceptual  constancy  concerns  the  kind 
of  visual  cues  that  may  be  used  for  distance  scal¬ 
ing.  In  particular,  there  is  some  evidence  that  pro¬ 
prioception  from  ocular  vergence  may  provide 
information  for  distance  scaling,  at  least  for  short 
viewing  distances  such  as  those  encountered  with 
HMDs.  For  example,  von  Hofsten  (1976)  investi¬ 
gated  perceived  distance  while  observers  viewed 
a  target  through  a  polarization  stereoscope  and 
found  that  perceived  distance  was  determined  by 
relative  differences  in  convergence  angle,  not 
by  absolute  convergence  angle.  Owens  and  Leib- 
owitz  (1976)  investigated  the  relationship  among 
dark  focus,  dark  vergence,  and  the  perceived  dis¬ 
tance  of  a  light  point  presented  in  the  dark.  The 
results  showed  that  perceived  distance  was  cor¬ 
related  with  dark  vergence  but  not  with  dark 
focus.  These  authors  suggested  that  vergence 
may  be  a  major  cue  to  distance  perception.  A  fol¬ 
low-up  study  by  Owens  and  Leibowitz  (1980) 
revealed  a  similar  correlation  between  perceived 
distance  and  vergence  using  an  adaptation  para¬ 
digm.  Wetzel,  Pierce,  and  Geri  (1996)  also  report¬ 
ed  a  strong  relationship  between  vergence  and 
perceived  size  when  viewing  objects  on  near  (0.6 1 
or  0.95  m)  and  far  (8  m)  real-image  displays. 

Thus  HMDs  that  cause  vergence  to  drift  toward 
a  dark  vergence  level,  such  as  low-contrast  or  dim 
displays,  may  cause  distance  to  be  misperceived. 
This,  in  turn,  may  cause  the  size,  depth,  or  speed  of 
HMD  imagery  to  be  misperceived  owing  to  inap¬ 
propriate  distance  scaling  (Patterson  &  Martin, 
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1992).  Moreover,  with  respect  to  the  use  of  mo¬ 
nocular  HMDs,  the  vergence  system  becomes 
open  loop  because  only  one  eye  sees  the  HMD 
imagery.  Under  such  conditions,  it  may  be  that 
perceived  distance  is  affected  in  complex  ways. 
Ellis,  Bucher,  and  Menges  (1995)  reported  that 
errors  of  judged  depth  of  virtual  objects  are  asso¬ 
ciated  with  variation  in  binocular  vergence. 

Moreover,  there  is  some  evidence  that  pro¬ 
prioception  from  accommodation  may  provide 
information  for  distance  scaling  and  that  mis- 
accommodation  may  affect  perceived  distance. 
Edgar,  Pope,  and  Craig  (1993)  reported  that  many 
people  misaccommodate  when  viewing  virtual 
displays  superimposed  upon  a  real  scene  and  that 
such  mis  accommodation  can  lead  to  misperceived 
distance  and  size.  (Note,  however,  that  Owens  & 
Leibowitz  [1976, 1980]  found  no  relation  between 
accommodation  and  perceived  distance.)  Peli 
(1990)  also  noted  that  the  perceived  size  of  image¬ 
ry  on  a  monocular,  occluding  HMD  appears  to 
change  depending  on  the  distance  of  the  surface  to 
which  the  opposite  eye  is  accommodated.  HMDs 
that  cause  accommodation  to  drift  toward  a  dark 
focus  level,  such  as  low-contrast  or  dim  displays, 
may  cause  perceived  distance  to  be  misperceived. 
This,  in  turn,  may  cause  the  size,  depth,  or  speed 
of  HMD  imagery  to  be  misperceived  owing  to  in¬ 
appropriate  distance  scaling  (Patterson  &  Martin, 
1992). 

Recommendations 

Perceptual  research  reveals  that  there  are  per¬ 
ceptual  consequences  related  to  low  contrast  and 
brightness  in  HMDs.  Specifically,  depth  of  held, 
dark  focus  and  dark  vergence,  and  perceptual  con¬ 
stancy  can  be  affected  by  low  contrast  and  bright¬ 
ness.  With  respect  to  perceptual  constancy,  it 
involves  a  cue  integration  process  that  could  affect 
the  perceived  size,  depth,  and/or  distance  of  ob¬ 
jects  presented  within  the  HMD  if  any  of  the  rel¬ 
evant  cues  are  misregistered  by  the  visual  system. 
Moreover,  dark  focus  and  dark  vergence  can  af¬ 
fect  HMD  visibility  as  well  as  produce  a  misreg¬ 
istration  of  oculomotor  cues,  which  may  affect 
perceptual  constancy.  Thus,  HMD  brightness  and 
contrast  should  be  sufficient  so  as  to  minimize  the 
tendency  for  accommodation  to  drift  toward  dark 
focus,  and  minimize  the  tendency  for  vergence  to 
drift  toward  dark  vergence,  and  at  the  same  time 


produce  a  relatively  large  depth  of  focus  so  that 
HMD  images  appear  sharp  and  in  focus  while 
the  user  is  viewing  the  outside  world  or  another 
display. 

With  respect  to  a  recommended  minimum  lu¬ 
minance  level,  vergence  appears  to  be  valid  down 
to  a  level  of  0.1  cd/m2  (Ivanoff,  1955).  Accom¬ 
modation,  however,  appears  to  be  valid  down  to 
a  level  of  somewhere  between  6.9  cd/m2  and 
342.6  cd/m2  (Johnson,  1976;  Leibowitz  &  Owens, 
1975b).  Thus,  the  accommodation  response  seems 
to  set  the  lower  limit  on  the  usable  luminance 
level  of  HMDs.  With  respect  to  a  recommended 
minimum  contrast  level,  a  Michelson  contrast  of 
at  least .  10  is  desirable  (Velger,  1998,  pp.  70-71). 
For  example,  when  imagery  presented  on  an  HMD 
is  seen  against  the  outside  world  under  daylight 
conditions,  the  luminance  of  the  image  source 
should  be  approximately  27,410. 1  cd/m2  (Velger, 
1998,  p.  90),  which  would  make  the  Michelson 
contrast  of  the  symbology  be  about .  10,  or  make 
the  contrast  ratio  be  about  1.2,  in  which  contrast 
is  defined  as  the  ratio  of  the  luminance  of  the  vir¬ 
tual  image  to  the  luminance  of  the  outside  world 
transmitted  through  a  combiner  (Velger,  1998, 
pp.  70-71).  However,  the  Video  Electronics  Stan¬ 
dards  Association  flat  panel  display  measure¬ 
ments  standard,  version  2.0,  2001,  recommends 
a  minimum  contrast  of  .25  to  maintain  visibility 
of  imagery. 

ACCOMMODATION- VERGENCE 
SYNERGY 

When  one  views  an  HMD,  the  stimulus  to  ac¬ 
commodation  would  be  the  imagery  presented  on 
the  surface  of  the  HMD.  However,  if  the  user 
changes  his  or  her  vergence  angle  to  view  a  target 
appearing  in  a  virtual  3-D  scene  presented  on  the 
HMD,  the  vergence  angle  can  be  mismatched  rel¬ 
ative  to  accommodative  demand  (Wann,  Ruston, 
&  Mon- Williams,  1995).  An  analogous  situation 
can  occur  when  a  monocular  HMD  is  worn  and 
the  imagery  is  fixated.  In  this  case,  the  accom¬ 
modative  stimulus  is  the  HMD  imagery  but  ver¬ 
gence  becomes  open  loop  because  the  imagery  is 
presented  to  only  one  eye. 

The  existence  of  an  accommodation-vergence 
mismatch  can  create  several  problems  for  the 
HMD  user  and  arise,  in  part,  from  the  strong  syn¬ 
ergy  and  coupling  between  accommodation  and 
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vergence  (Toates,  1972, 1974).  An  accommodation- 
vergence  mismatch  can  create  eyestrain  or  visual 
discomfort,  a  frequently  reported  problem  with 
HMD  use  (Mon- Williams,  Wann,  &  Ruston,  1993; 
Ruston  et  al.,  1994;  Shibata,  2002;  Velger,  1998). 
Moreover,  an  accommodation- vergence  mismatch 
can  cause  the  HMD  imagery  to  become  blurred 
because  vergence  would  be  driving  accommoda¬ 
tion  to  respond  to  a  focal  distance  that  would  be 
different  from  the  distance  at  which  the  stimulus 
for  accommodation  is  established.  Finally,  an  ac¬ 
commodation- vergence  mismatch  may  alter  the 
perception  of  distance,  size,  depth,  or  speed  of  ob¬ 
jects  presented  on  the  HMD  for  reasons  discussed 
previously  relating  to  perceptual  constancy.  Re¬ 
call  that  the  visual  system  likely  scales  retinal  size, 
binocular  disparity,  and  perhaps  retinal  speed  in 
accordance  with  viewing  distance  information,  a 
process  called  distance  scaling.  If  proprioception 
from  ocular  vergence  and/or  accommodation  pro¬ 
vides  information  about  viewing  distance  (Edgar 
et  al.,  1993;  Ellis  et  al.,  1995;  Owens  &  Leibowitz, 
1976, 1980;  von  Hofsten,  1976;  Wetzel  et  al.,  1996), 
then  an  accommodation- vergence  mismatch  may 
disrupt  the  distance  scaling  operation  and  cause 
the  size,  depth,  or  speed  of  HMD  imagery  to  be 
altered  (Patterson  &  Martin,  1992). 

Sheedy  and  Bergstrom  (2002),  who  had  partic¬ 
ipants  perform  several  text-based  tasks  as  well  as 
a  routine  of  head  and  body  movements,  compared 
five  types  of  displays:  monocular  head-mounted, 
binocular  head-mounted,  hard  copy,  flat  panel, 
and  a  small-format  portable  display.  Performance 
speed  on  the  text-based  tasks  with  HMDs  was 
generally  comparable  to  performance  on  the  other 
displays,  but  symptoms  of  eyestrain  and  blurry 
vision  were  significantly  higher  with  the  former 
displays.  Those  authors  also  found  that  motion- 
related  symptoms  with  the  HMDs  were  not  sig¬ 
nificantly  different  from  those  with  the  other 
displays.  Unlike  many  past  studies,  theirs  found 
that  performance  and  comfort  on  the  HMDs  were 
more  similar  to  those  with  traditional  displays, 
which  was  likely  attributable  in  part  to  concor¬ 
dance  of  the  accommodative  and  vergence  stimuli 
(i.e.,  the  tasks  did  not  induce  the  observers  to  sig¬ 
nificantly  change  convergence  angle). 

Recommendations 

An  accommodation-vergence  mismatch  can 
affect  the  perceived  size,  depth,  and  distance  of 


objects  presented  within  an  HMD  because  of  the 
operation  of  perceptual  constancy.  Such  a  mis¬ 
match  can  also  create  eyestrain  and  discomfort. 
These  problems  will  be  less  for  tasks  that  do  not 
induce  the  user  to  significantly  change  conver¬ 
gence  angle;  the  problems  will  be  greater  when 
objects  in  a  virtual  scene  are  displayed  in  depth 
and  the  user  is  induced  to  change  convergence 
angle.  More  research  is  needed  to  determine  the 
exact  limits  of  tolerance  for  accommodation- 
vergence  mismatch. 

FIELD  OF  VIEW 

The  field  of  view  of  an  HMD  is  important  for 
the  obvious  reason,  other  factors  being  constant, 
that  a  larger  field  of  view  will  better  simulate  nat¬ 
ural  viewing,  given  that  the  field  of  view  of  the 
human  visual  system  extends  200°  horizontal  by 
130°  vertical,  with  the  central  120°  being  the  area 
of  binocular  overlap  (Velger,  1998,  p.  50).  Wide 
fields  of  view  can  be  created  with  relatively  large 
optics  (which  increases  the  weight  of  the  system) 
or  smaller  eye  relief  (Velger,  1998,  pp.  64—65). 
However,  increasing  field  of  view  will  decrease 
display  resolution  because  the  same  pixels  are 
mapped  to  a  larger  display  area  (Fisher,  1994). 

A  large  field  of  view  will  produce  a  greater 
sense  of  immersion  (Primeau,  2000)  as  well  as 
provide  the  stimulus  conditions  necessary  for  ade¬ 
quate  visual  functioning  (also  see  S.  P.  Rogers, 
Asbury,  &  Szoboszlay,  2003).  Accounting  for 
this  is  the  fact  that  the  visual  system  is  anatomi¬ 
cally  subdivided  into  two  different  visual  path¬ 
ways,  one  predominately  activated  by  stimulation 
in  the  central  portion  of  the  retina  and  the  other 
activated  by  stimulation  in  the  central  retina  and 
periphery.  Thus,  a  restricted  field  of  view  may  ac¬ 
tivate,  or  fail  to  activate,  these  two  pathways  in  an 
unnatural  way.  In  order  to  consider  the  implica¬ 
tions  of  a  restricted  field  of  view,  we  now  turn  to 
a  brief  discussion  of  these  two  visual  pathways. 

Visual  Pathways 

The  visual  system  is  composed  of  the  parvo- 
cellular  and  magnocellular  pathways  (e.g.,  Liv¬ 
ingstone  &  Hubei,  1988;  Schiller,  Logothetis,  & 
Charles,  1990).  The  parvocellular  pathway  has 
connections  mainly  from  the  central  retina,  and  it 
projects  to  areas  in  the  visual  cortex  that  make  up 
the  ventral  cortical  stream.  Areas  in  the  ventral 
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stream  functionally  analyze  spatial  pattern  infor¬ 
mation.  Cells  in  these  cortical  areas  have  a  sus¬ 
tained  or  sluggish  response  and  poor  temporal 
acuity.  Many  cells  in  these  areas  respond  to  color 
information,  and  many  cells  respond  to  fine  detail 
and  high  spatial  frequency  information.  The  ven¬ 
tral  stream  is  thought  to  be  involved  in  the  func¬ 
tional  analysis  of  spatial  pattern  information  for 
the  purpose  of  identifying  objects. 

The  magnocellular  pathway  has  connections 
from  the  central  and  peripheral  retina,  and  it  pro¬ 
jects  to  areas  in  the  visual  cortex  that  make  up  the 
dorsal  cortical  stream.  Areas  in  the  dorsal  stream 
functionally  analyze  optic  flow  information  for 
heading  control  (Peuskens,  Sunaert,  Dupont,  Van 
Hecke,  &  Orban,  2001)  and  biological  motion 
(Grossman  &  Blake,  2001 ;  Grossman  et  al.,  2000), 
and  they  integrate  vision  with  action  (e.g.,  T’so 
&  Roe,  1995;  Van  Essen  &  DeYoe,  1995;  Yabuta, 
Sawatari,  &  Callaway,  2001).  Cells  in  these  cor¬ 
tical  areas  have  a  transient  response  and  high  tem¬ 
poral  acuity.  Cells  in  these  areas  do  not  respond 
to  color  information  but  do  respond  to  coarse  de¬ 
tail  and  relatively  low  spatial  frequency  informa¬ 
tion.  The  dorsal  stream  is  thought  to  be  involved 
in  the  functional  analysis  of  motion  information 
for  the  purpose  of  determining  spatial  relations 
and  controlling  heading  during  locomotion. 

Milner  and  Goodale  (1995)  made  an  even  great¬ 
er  distinction  between  the  two  pathways.  They 
claimed  that  the  ventral  stream  is  involved  in  the 
representation  of  visual  experience,  whereas  the 
dorsal  stream  is  involved  in  action  priming  and 
motor  control.  This  view  implies  that  information 
gleaned  from  research  on  visual  judgments  and 
reports,  which  would  involve  ventral  stream  func¬ 
tioning,  would  not  necessarily  generalize  to 
sensory-motor  control  tasks,  which  would  entail 
dorsal  stream  functioning. 

Restricted  Field  of  View 

Turning  back  to  the  issue  of  field  of  view,  using 
an  HMD  with  a  restricted  field  of  view  would  mean 
that  stimulation  would  occur  mainly  in  the  central 
portion  of  the  retinae  of  the  two  eyes.  Although 
eye  movements  usually  would  be  possible  with  an 
HMD,  it  is  likely  that  the  user  would  spend  much 
of  the  time  looking  in  the  forward  direction.  This, 
in  turn,  would  mean  that  the  ventral  stream,  and 
not  the  dorsal  stream,  would  be  predominantly  ac¬ 
tivated.  Under  such  conditions,  an  observer’s  abil¬ 


ity  to  control  heading  or  process  spatial  orienta¬ 
tion  might  be  compromised,  as  might  be  the  sense 
of  immersion. 

For  example,  Allison,  Howard,  and  Zacher 
(1999)  investigated  the  effect  of  the  size  of  the 
field  of  view  on  roll  vection  and  illusory  self-tilt  in 
a  tumbling  room.  They  found  that  complete  360° 
body  rotation  (tumbling)  and  increased  speed  of 
self-rotation  were  perceived  by  most  observers 
under  a  full-field  condition.  For  smaller  fields  of 
view,  tilt  or  partial  tumbling  and  slower  speeds 
of  self-rotation  were  perceived.  When  the  field  of 
view  was  50°,  one  half  of  all  observers  reported 
tumbling  at  an  intermediate  speed.  This  suggests 
that  a  field  of  view  of  at  least  50°  is  needed  for 
some  sense  of  immersion  and  that  a  larger  field 
of  view  would  be  needed  for  a  complete  sense  of 
immersion. 

Of  course,  the  required  size  of  field  of  view  for 
an  HMD  would  depend  upon  the  application  un¬ 
der  consideration.  In  a  study  involving  a  simulat¬ 
ed  nighttime  attack,  Osgood  and  Wells  (1991) 
examined  size  of  the  field  of  view  of  the  image 
from  an  aircraft-fixed  sensor  seen  on  a  head-up 
display  (HUD)  and  for  a  head- steered  sensor  seen 
on  an  HMD.  With  the  HMD,  observers  acquired 
the  targets  quicker  up  to  a  field  of  view  of  40°,  be¬ 
yond  which  performance  did  not  significantly 
change.  Kenyon  and  Kneller  (1993)  investigated 
the  effect  of  field  of  view  on  a  visual  tracking  task 
and  found  that  near  asymptotic  performance  was 
obtained  with  a  field  of  view  of  only  40°  (also  see 
Jennings  &  Craig,  2000).  Note  that  these  studies 
involved  tasks  that  likely  depended  upon  activa¬ 
tion  primarily  of  the  ventral  stream. 

Other  studies  have  involved  tasks  that  likely 
depended  upon  activation  of  the  dorsal  stream. 
For  example,  Brickner  and  Foyle  ( 1 990)  reported 
that  field  of  view  affected  heading  control  in  a 
flight  simulator,  with  more  narrow  fields  of  view 
(e.g.,  25°)  leading  to  impaired  performance  rela¬ 
tive  to  larger  fields  of  view  (e.g.,  55°).  Weikhorst 
and  Vacarro  (1988)  found  that  maneuvers  per¬ 
formed  by  pilots  in  a  flight  simulator,  such  as  the 
aileron  and  barrel  roll,  were  performed  signifi¬ 
cantly  better  with  a  very  wide  field  of  view  of 
300°  than  with  a  field  of  view  of  36°  or  144°.  They 
also  found  that  the  accuracy  in  dropping  bombs 
improved  when  the  field  of  view  was  larger.  In  a 
study  involving  low-altitude  flight,  Kruk  and 
Runnings  (1989)  varied  field  of  view  from  87°  to 


564 


Fall  2006  -  Human  Factors 


127°  and  found  that  the  subjective  workload  of 
pilots  was  substantially  higher  with  a  limited  held 
of  view.  Moreover,  significant  increases  in  maneu¬ 
vering  time  were  found  for  formation  hying  as 
held  of  view  was  decreased,  presumably  because 
of  a  lack  of  motion  cuing  and  a  reduced  horizon 
reference.  Finally,  for  a  binocular  HMD  involving 
OTW  viewing,  a  held  of  view  greater  than  50° 
was  required  for  creating  a  sense  of  immersion 
(Clapp,  1987;  Fisher  1994;  Velger,  1998,  p.  65). 

In  a  study  involving  a  large  non- HMD  display, 
Duh,  Lin,  Kenyon,  Parker,  and  Furness  (2001) 
found  that  observers  showed  greater  amounts  of 
postural  disturbance  with  increasing  held  of  view, 
up  to  a  held  of  view  of  1 80°,  the  largest  value  test¬ 
ed.  In  a  related  study  also  involving  a  non-HMD 
display,  Lin,  Duh,  Parker,  Abi-Rached,  and  Furness 
(2002)  found  that  observers  reported  higher  levels 
of  simulator  sickness,  as  well  as  higher  levels  of 
“presence,”  with  increased  held  of  view,  up  to  a 
held  of  view  of  140°.  These  results  suggest  that 
a  large  held  of  view  is  needed  for  creating  a  sense 
of  immersion,  which  in  turn  is  related  to  a  greater 
potential  for  simulator  sickness  and  postural 
instability.  This  is  likely  because  a  larger  held  of 
view  engages  more  of  the  dorsal  stream,  which 
would  be  responsible  for  postural  control  and  body 
alignment  in  space,  and  thus  a  greater  potential  is 
created  for  a  mismatch  between  the  vestibular 
system  and  the  visual  system. 

Melzer  (1998)  suggested  several  techniques  for 
creating  HMDs  with  a  large  held  of  view  as  well 
as  high  resolution.  These  techniques  include  de¬ 
signing  high-resolution  areas  of  interest  or  imple¬ 
menting  a  partial  binocular  overlap  design. 

Recommendations 

The  functioning  of  the  parvocellular  and  mag- 
nocellular  pathways  and  the  ventral  and  dorsal 
cortical  streams  determines,  in  part,  the  type  of 
visual-motor  tasks  that  an  individual  can  perform. 
Thus,  knowledge  of  these  pathways  may  enable 
an  individual  to  better  predict  the  type  of  task  that 
may  be  impaired  with  a  certain  restricted  held  of 
view.  For  example,  for  successful  performance  of 
activities  or  tasks  involving  ventral  stream  func¬ 
tioning,  such  as  targeting  and  object  recognition, 
a  held  of  view  as  small  as  40°  may  be  sufficient. 
For  successful  performance  of  activities  or  tasks 
involving  dorsal  stream  functioning,  such  as  visu¬ 
al  orientation  to  peripheral  stimulation,  control¬ 


ling  the  position  of  a  moving  vehicle  relative  to 
the  position  of  other  vehicles  (e.g.,  formation  hy¬ 
ing),  or  perceiving  a  full  sense  of  immersion,  a 
held  of  view  much  greater  than  60°  would  likely 
be  needed.  Kruk  and  Runnings  (1989),  for  exam¬ 
ple,  stated  that  a  held  of  view  of  127°  is  required 
for  flight  simulation  applications.  However,  a  large 
held  of  view  may  create  problems  related  to  sim¬ 
ulator  sickness  and/or  postural  instability.  More¬ 
over,  achieving  both  a  wide  held  of  view  and  high 
resolution  is  difficult,  given  current  technological 
limitations.  The  ideas  discussed  in  this  paper  pro¬ 
vide  information  to  help  establish  the  relative 
design  priorities  for  different  HMD  applications. 
If  accurate  perception  of  self-motion  is  required, 
for  example,  it  may  be  prudent  to  sacrihce  dis¬ 
play  resolution  to  gain  a  greater  held  of  view. 

BINOCULAR  INPUT 

Under  natural  viewing  conditions,  the  two  eyes 
usually  view  a  common  region  of  visual  space, 
albeit  with  horizontal  binocular  parallax  (Howard 
&  Rogers,  1995, 2002).  However,  with  HMDs,  the 
potential  exists  for  the  two  eyes  to  receive  very 
different  input.  When  a  monocular  HMD  is  worn, 
for  example,  one  eye  views  the  HMD  symbology 
while  the  other  eye  views  a  real-world  scene,  thus 
possibly  creating  interocular  differences  in  bright¬ 
ness,  contrast,  color,  dicker,  shape,  size,  motion, 
and/or  accommodative  demand  (Velger,  1998,  p. 
25).  When  a  biocular  or  binocular  HMD  is  worn, 
optical  misalignment  and/or  image  distortions 
between  the  two  eyes’  views  can  create  such  inte¬ 
rocular  differences  as  well. 

Binocular  Rivalry 

In  many  cases,  interocular  differences  in  image 
characteristics  can  disrupt  binocular  vision  by 
creating  the  conditions  for  binocular  rivalry. 
Binocular  rivalry  (e.g.,  Blake,  1989;Breese,  1899; 
Howard,  2002;  Howard  &  Rogers,  1995;  Levelt, 
1965)  refers  to  a  state  of  competition  between  the 
eyes,  such  that  one  eye  inhibits  the  visual  process¬ 
ing  of  the  other  eye.  The  visibility  of  the  images  in 
the  two  eyes  fluctuates,  with  one  eye’s  view  be¬ 
coming  visible  while  the  other  eye’s  view  is  ren¬ 
dered  invisible  and  suppressed,  which  reverses 
over  time;  the  images  projected  to  the  two  eyes  are 
rarely  visible  simultaneously. 

The  inhibition  evoked  by  binocular  rivalry 
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likely  occurs  at  a  number  of  levels  throughout  the 
visual  system  (Meng,  Chen,  &  Qian,  2004),  and 
it  continues  at  a  relatively  constant  rate  during  the 
course  of  suppression  (Norman,  Norman,  & 
Bilotta,  2000).  Binocular  rivalry  makes  visual  pro¬ 
cessing  unstable  and  unpredictable.  For  example, 
Schall,  Nawrot,  Blake,  and  Yu  (1993)  showed 
that  binocular  rivalry  suppression  impairs  the 
ability  of  observers  to  visually  guide  and  direct 
attention  to  targets  in  the  visual  field. 

This  phenomenon  of  inattention  and  rivalry 
may  be  related  to  the  phenomenon  of  inattention- 
al  blindness,  wherein  unexpected  objects  go  un¬ 
noticed  because  attention  is  directed  elsewhere 
(Mack  &  Rock,  1998;  Mack,  Tang,  Tuma,  Kahn, 
&  Rock,  1992;  Rock,  Linnett,  Grant,  &  Mack,  1992; 
Rock  &  Mack,  1994),  or  change  blindness,  wherein 
changes  in  objects  go  unnoticed  because  attention 
is  directed  elsewhere  (e.g.,  Rensink,  O’Regan,  & 
Clark,  1995).  These  phenomena  (see  Palmer, 
1999,  pp.  534-540  for  discussion)  are  important 
because  they  represent  visual  processes  by  which 
a  person  may  miss  information  or  signals  while 
using  an  HMD. 

Hakkinen  (2003)  stated  that  monocular  HMDs 
produce  a  common  problem  of  rivalry  and  per¬ 
ceptual  instability  and  should  be  used  only  briefly 
(1-5  min).  This  claim,  however,  was  made  with 
regard  to  closed  or  occluded  monocular  HMDs, 
and  it  is  not  known  whether  rivalry  would  be  such 
a  problem  with  open  or  semitransparent  monoc¬ 
ular  HMDs. 

With  respect  to  minimizing  the  occurrence  of 
binocular  rivalry  when  HMDs  are  worn,  it  may 
be  that  binocular  fusion  (i.e.,  sensory  blending  of 
the  two  eyes’  views)  can  mitigate  against  rivalry. 
For  example,  with  a  semitransparent  or  open 
monocular  HMD,  the  background  scene  would  be 
viewed  by  both  eyes,  and  binocular  fusion  of  the 
background  may  mitigate  against  the  binocular 
rivalry  that  could  occur  between  the  imagery  in  the 
eye  that  sees  the  HMD  and  the  background  scene 
viewed  by  the  other  eye.  Blake  and  Boothroyd 
(1985)  showed  that  when  one  eye  views  a  set  of 
vertical  contours  while  the  other  eye  simultane¬ 
ously  views  a  set  of  vertical  contours  (which 
should  elicit  binocular  fusion)  and  a  set  of  hori¬ 
zontal  contours  (which  should  provoke  binocular 
rivalry),  the  result  is  binocular  fusion,  not  rivalry. 
The  presence  of  matching  features  in  the  two 
eyes  seemed  to  make  those  features  exempt  from 


rivalry.  Thus,  it  is  possible  that  rivalry  would  not 
be  a  significant  problem  when  monocular  semi¬ 
transparent  HMDs  are  worn  if  the  background 
scene  viewed  by  both  eyes  is  easily  fusible.  How¬ 
ever,  the  results  of  one  study  suggest  that  even 
under  conditions  for  which  fusion  is  possible, 
binocular  rivalry  may  still  be  a  significant  prob¬ 
lem.  Laramee  and  Ware  (2002)  found  that  re¬ 
sponse  times  were  significantly  greater  on  a  table 
look-up  task  when  the  table  was  presented  on  a 
transparent  monocular  HMD  (30%  transmissive) 
while  observers  binocularly  viewed  dynamic 
background  imagery  presented  on  a  television. 

Even  with  interocular  differences  in  image 
characteristics  that  can  be  binocularly  fused,  un¬ 
wanted  perceptual  effects  can  still  occur.  For  ex¬ 
ample,  horizontal  or  vertical  size  disparities  (i.e., 
size  differences  between  the  horizontal  or  vertical 
extent  of  corresponding  images  in  the  two  eyes) 
can  create  the  erroneous  perception  of  a  slanted 
surface  in  depth,  which  can  perceptually  interact 
with  zero-disparity  stimuli  (Ogle,  1938;  Pierce, 
Arrington,  &  Moreno,  1999;  Pierce  &  Howard, 
1997). 

Partial  Overlap  HMDs 

In  some  applications  in  which  biocular  or  bin¬ 
ocular  HMDs  are  used,  having  a  sufficient  field 
of  view  is  important  for  a  sense  of  immersion.  To 
increase  the  width  of  the  field  of  view  with  biocu¬ 
lar  or  binocular  HMDs,  such  HMDs  can  be  con¬ 
vergent  or  divergent  partial  overlap  displays.  Such 
displays  have  a  central  field  of  binocular  overlap 
and  peripheral  side  regions  of  monocular  viewing 
(Velger,  1998,  p.  56).  Specifically,  convergent  dis¬ 
plays  have  the  right  eye  view  the  area  to  the  left 
of  the  central  region  and  the  left  eye  view  the  area 
to  the  right  of  the  central  region.  Divergent  dis¬ 
plays  have  the  right  eye  view  the  area  to  the  right 
of  the  central  region  and  left  eye  view  the  area  to 
the  left  of  the  central  region. 

The  convergent  and  divergent  partial  overlap 
displays  can  produce  perceptual  conflicts  such  as 
luning,  which  refers  to  a  subjective  darkening  of 
the  monocular  regions  near  the  border  with  the 
binocular  overlap  region;  it  is  possible  that  lun¬ 
ing  is  related  to  binocular  rivalry  (Levelt,  1965; 
Velger,  1998,  p.  56).  Luning  may  interfere  to  some 
degree  in  tasks  such  as  target  detection.  For  exam¬ 
ple,  Kruk  and  Longridge  (1984)  found  that  target 
detection  was  degraded  within  a  5°  area  near  the 
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binocular  overlap  regions  for  both  a  25°  overlap 
condition  and  a  45°  degree  overlap  condition. 
Luning  can  be  minimized  by  using  a  convergent 
display  that  has  reduced  luminance  near  the 
edges  of  the  binocular  region  (Grigby  &  Tsou, 
1994;  Klymenko  et  al.,  1994a,  1994b;  Melzer  & 
Moffitt,  1991;  Rash,  Mozo,  Me  Entire  &  Licina, 
1996;  Velger,  1998,  p.  56). 

Luning  may  also  be  related  to  an  interocular 
suppression  that  affects  unpaired  monocular 
images  (called  half  occlusions),  which  do  not 
normally  occur  in  the  real  world  (Nakayama 
&  Shimojo,  1990).  For  example,  with  binocular 
viewing  of  a  surface  located  behind  an  aperture 
(e.g.,  window),  unpaired  images  will  exist  in  one 
eye  that  do  not  exist  in  the  other  eye.  These  half 
occlusions  will  either  undergo  visual  suppression 
or  be  perceived  in  a  given  depth  location,  depend¬ 
ing  upon  whether  the  stimulus  arrangement  is 
consistent  with  the  physics  of  occlusion.  Luning 
may  be  a  result  of  having  unpaired  images  in  one 
or  both  eyes  that  are  inconsistent  with  the  physics 
of  occlusion. 

In  a  study  comparing  HMDs  with  full  binocu¬ 
lar  overlap  to  those  with  partial  binocular  overlap 
(convergent  or  divergent),  Klymenko,  Harding, 
Beasley,  and  Martin  (1999)  reported  that  response 
times  were  fastest,  and  accuracy  highest,  in  a  tar¬ 
get  acquisition  task  when  HMDs  with  full  binoc¬ 
ular  overlap  were  used;  response  times  were 
slowest,  and  accuracy  lowest,  when  HMDs  with 
divergent  partial  overlap  were  employed;  response 
times  and  accuracy  were  intermediate  when 
HMDs  with  convergent  partial  overlap  were  used. 

Tolerance  Levels 

Even  with  HMDs  with  full  binocular  overlap, 
the  imagery  projected  to  the  two  eyes  should 
have  the  same  or  similar  spatial  characteristics; 
otherwise  binocular  rivalry  may  be  provoked. 
With  regard  to  visual  tolerance  for  interocular  dif¬ 
ferences  in  stimulation  using  a  binocular  HMD, 
Rash  et  al.  (1996)  suggested  that  the  following 
interocular  differences  should  be  tolerable:  a  dif¬ 
ference  in  luminance  of  up  to  30%,  a  rotational 
difference  of  up  to  10  aremin,  horizontal  or  ver¬ 
tical  differences  in  image  size  of  up  to  1.5%,  and 
deviation  between  centers  of  the  two  displays  of 
0.18  prism  diopters.  According  to  Tsou  and  Shenker 
(2000),  the  tolerance  levels  for  misalignment  er¬ 
rors  for  a  closed  binocular  HMD  are  ±23  aremin 


horizontal,  ±1 1.5  aremin  vertical,  and  ±12  aremin 
cyclorotational.  For  a  semitransparent  binocular 
HMD,  the  tolerance  levels  are  ±10  aremin  hori¬ 
zontal,  ±4  aremin  vertical,  and  ±6  aremin  cycloro¬ 
tational. 

With  respect  to  interocular  differences  in  lumi¬ 
nance,  Levelt  (1965)  reviewed  the  literature  on 
binocular  brightness  summation  as  well  as  binoc¬ 
ular  rivalry.  He  reported  that  the  luminance  levels 
in  the  two  eyes  can  usually  be  fused  provided  that 
the  contrasts  are  not  reversed  and  provided  that  the 
brightness  differences  are  not  too  large.  He  sug¬ 
gested  that  the  binocular  combination  process 
involves  a  relative  weighting  of  the  two  eyes’  in¬ 
put,  with  the  presence  of  contours  in  a  given  eye 
increasing  the  weight  of  the  coefficient  for  that  eye. 

Recommendations 

The  characteristics  of  binocular  vision  can  af¬ 
fect  the  use  of  HMDs.  For  example,  half  occlusions 
may  affect  the  viewing  of  partially  overlapped 
HMDs.  Moreover,  the  binocular  fusion  of  an  OTW 
scene  may  mitigate  against  the  occurrence  of  bin¬ 
ocular  rivalry,  which  may  involve  suppression  at 
different  levels  of  the  visual  system.  Thus,  it  is 
important  to  create  viewing  conditions  that  pro¬ 
mote  binocular  fusion,  which  requires  that  the  two 
eyes  view  the  same  or  very  similar  images  and 
scenes.  With  respect  to  semitransparent  monoc¬ 
ular  HMDs,  research  should  investigate  whether 
binocular  fusion  of  the  background  scene  that  is 
seen  by  both  eyes  mitigates  against  the  rivalry  that 
could  occur  between  the  imagery  in  the  eye  that 
sees  the  HMD  and  the  background  scene  viewed 
by  the  other  eye.  With  regard  to  biocular  or  binoc¬ 
ular  HMDs,  the  human  visual  system  appears  to 
be  tolerant  to  interocular  differences  in  luminance 
of  no  more  than  30%,  rotational  differences  of  no 
more  than  10  aremin,  horizontal  or  vertical  dif¬ 
ferences  in  image  size  of  no  more  than  1.5%,  and 
deviation  between  centers  of  the  two  displays  of 
no  more  than  0. 18  prism  diopters. 

HEAD  MOVEMENTS 

Under  natural  viewing  conditions,  head  move¬ 
ments  create  shifts  or  displacements  in  the  posi¬ 
tion  of  retinal  images.  For  example,  if  an  observer 
voluntarily  moves  his  or  her  head  to  the  right 
(without  an  eye  movement),  objects  that  were 
positioned  directly  in  front  of  the  observer  now 
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appear  located  in  the  left  visual  hemifield,  accom¬ 
panied  by  a  rightward  shift  (from  the  foveae)  of 
the  retinal  images  in  the  two  eyes.  With  an  HMD, 
head  movements  do  not  create  shifts  in  the  posi¬ 
tion  of  retinal  images  because  the  visual  display 
moves  with  the  head.  In  this  case,  if  an  observer 
wearing  an  HMD  moves  his  or  her  head  to  the 
right,  displayed  objects  that  appeared  to  be  locat¬ 
ed  directly  in  front  of  the  observer  will  continue  to 
appear  to  be  so  located.  To  mimic  natural  viewing 
(i.e.,  employ  conformal  imagery),  the  displayed 
imagery  could,  in  principle,  be  updated  and  shift¬ 
ed  leftward  on  the  visual  display  by  computer, 
which,  in  turn,  would  shift  the  retinal  images  right- 
ward.  To  do  so,  however,  head  position  would 
need  to  be  tracked. 

In  addition  to  voluntary  head  movements,  in¬ 
voluntary  head  movements  may  also  need  to  be 
tracked.  Involuntary  head  movements  occur  when 
the  HMD  is  used  in  an  environment  involving  ac¬ 
celeration  and/or  vibration,  a  topic  discussed  at 
length  by  Velger  ( 1998).  In  such  an  environment, 
involuntary  head  movements  elicit  reflexive  eye 
movements  called  the  vestibulo-ocular  reflex 
(VOR).  The  VOR  compensates  for  the  involuntary 
head  movements  so  that  fixation  is  maintained  on 
a  given  object  and  the  image  is  stabilized  on  the 
retina.  For  instance,  if  an  individual’s  head  vi¬ 
brates  upward,  the  position  of  a  given  object  would 
move  downward  relative  to  the  eye  and  the  retinal 
image  would  slip  upward.  In  this  case,  the  VOR 
would  elicit  a  compensatory  downward  eye  move¬ 
ment  to  maintain  fixation  on  the  object  during  the 
head  movement.  In  the  real  world,  such  a  com¬ 
pensatory  eye  movement  system  is  important  for 
maintaining  fixation  on  objects  during  locomotion 
(Benson  &  Barnes,  1978;  Velger,  1998,  p.  207). 
When  an  HMD  is  worn,  however,  the  VOR  eye 
movements  disrupt  vision  because,  during  the 
involuntary  head  movement  produced  by  vibra¬ 
tion,  the  retinal  image  moves  with  the  head.  With 
HMDs,  the  compensatory  eye  movements  are  in¬ 
appropriate  (Lee  &  King,  1971 ;  Velger,  1998,  pp. 
208,232). 

Thus,  it  is  not  surprising  that  visual  perfor¬ 
mance  declines  with  head  vibration.  For  example, 
visual  acuity  decreases,  and  errors  in  aiming  in¬ 
crease,  by  as  much  as  a  factor  of  10  during  head 
vibration.  The  worst  performance  occurs  at  a  fre¬ 
quency  range  of  3.2  to  5  Hz  (Velger,  1998,  p.  216; 
Wells  &  Griffin,  1987a,  1987b,  1988);  below  1  Hz, 


smooth  pursuit  eye  movements  can  compensate 
for  the  VOR  (Huddleston,  1970;  Velger,  1998,  p. 
216).  One  potential  remedy  for  an  inappropriate 
VOR  response  would  be  to  measure  the  involun¬ 
tary  head  movements  with  a  head  tracker  and  shift 
the  image  on  the  display  in  the  same  direction 
as  the  VOR  (opposite  the  head  movement),  which 
would  stabilize  the  retinal  image  (Velger,  1998, 
p.  225;  Young,  1976).  Draper  et  al.  (1997)  sug¬ 
gested  that  one  cause  of  simulator  sickness  is  the 
inability  of  the  VOR  to  adapt  to  vestibular- visual 
mismatches. 

Head  Tracking 

When  an  HMD  is  worn,  the  user’s  head  posi¬ 
tion  needs  to  be  tracked  for  two  reasons:  to  mimic 
natural  viewing  while  making  voluntary  head 
movements  and  to  maintain  clear  vision  while 
working  in  an  environment  involving  acceleration 
and/or  vibration  and,  therefore,  involuntary  head 
movements.  Yet  head  tracking  is  difficult,  in  part, 
because  it  can  be  relatively  inaccurate  owing  to 
the  amplitude  and  frequency  of  head  movements 
(Velger,  1998,  p.  224;  Wells  &  Griffin,  1984, 
1987a,  b).  For  example,  voluntary  head  move¬ 
ments  may  exceed  angular  velocities  of  several 
hundreds  of  degrees  per  second  and  accelerations 
of  up  to  several  thousand  degrees  per  second 
squared  (Wells  &  Haas,  1990).  However,  these 
values  are  at  the  extreme  end  of  head  movement 
velocities  and  accelerations;  for  example,  Rash 
et  al.  (1998)  found  that  97%  of  all  head  move¬ 
ments  of  AH-64  pilots  were  between  0°  and  120°/s. 
Based  on  these  and  other  studies,  several  authors 
(Velger,  1998,  pp.  227, 232;  see  also  Ljung,  Morf, 
&  Falconer,  1978;  Merhav  &  Velger,  1991)  have 
recommended  that  head  position  computations 
be  carried  out  at  a  rate  of  120  to  240  Hz. 

The  temporal  delay  or  lag  between  actual  head 
position  and  update  of  the  displayed  imagery  can 
seriously  affect  the  use  of  HMDs  (Keller  &  Colucci, 
1998).  Temporal  delays  or  lags  in  the  computation 
of  head  position,  or  in  the  update  of  the  displayed 
imagery,  can  impair  human  performance  and  cre¬ 
ate  disorientation,  nausea,  and  discomfort  (Grun- 
wald  &  Kohn,  1994;  So  &  Griffin,  1992;  Velger, 
1998,  pp.  145, 171).  This  is  attributable,  in  part,  to 
a  sensory  conflict  between  proprioception  and 
vision  (Long  &  Wickens,  1994;  So  &  Griffin,  1992; 
Velger,  1998,  pp.  145,  171). 

Despite  the  general  agreement  concerning  the 
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negative  effects  of  temporal  delay  or  lag  in  pro¬ 
ducing  impaired  performance  and  discomfort, 
there  is  little  agreement  as  to  the  critical  delay  or 
lag  beyond  which  impairment  can  be  expected. 
According  to  So  and  Griffin  ( 1992),  the  delay  be¬ 
tween  head  movement  and  imagery  update  needs 
to  be  less  than  80  ms;  according  to  Padmos  and 
Milders  (1992),  the  delay  should  be  less  than  40 
ms;  and  according  to  Keller  and  Colucci  (1998), 
the  delay  should  be  less  than  16  ms. 

Jagacinski  and  Flach  (2003,  p.  99)  noted  that 
phase  lags  set  up  a  stability  limit  on  the  forward 
loop  gain  of  a  control  system,  with  larger  delays 
producing  a  decreasing  range  of  gains  that  will 
yield  stable  control.  With  long  delays,  there  may 
be  no  stable  gain  that  produces  a  rapid  head  track¬ 
er  response,  and  a  proportional  control  system 
may  not  work. 

To  shorten  the  temporal  delays  or  lags,  one 
may  need  to  increase  the  sampling  rate  or  com¬ 
putation  rate  of  head  position,  use  a  prediction  al¬ 
gorithm  to  predict  head  position  at  some  future 
point  in  time,  or  use  auxiliary  acceleration  mea¬ 
surements  (Merhav  &  Velger,  1991;Velger,  1998, 
p.  172).  Olano,  Cohen,  Mine,  and  Bishop  (1995) 
discussed  techniques  for  combating  latency  by 
making  the  rendering  latency  equal  one  National 
Television  Standards  Committee  field  (16.7  ms) 
and  by  rendering  the  pixels  in  a  scanned  display 
based  on  position  in  the  scan.  Various  types  of 
head-tracking  technology,  including  prediction 
algorithms  and  noise  filtering,  have  been  discussed 
by  Melzer  and  Moffitt  (1997)  and  by  Velger 
(1998). 

The  lack  of  agreement  in  the  literature  regard¬ 
ing  the  critical  delay  beyond  which  impairment 
can  be  expected  is  troublesome.  Part  of  the  reason 
for  the  disagreement  may  be  that  different  studies 
have  measured  different  aspects  of  visual  percep¬ 
tion  (recall  the  previous  discussion  of  parvocellu- 
lar  versus  magnocellular  pathways),  each  of  which 
may  have  possessed  a  different  critical  duration. 
What  is  needed  are  an  understanding  of  what 
kind  of  visual  events  are  occurring  when  a  head 
movement  is  performed,  voluntary  or  involuntary, 
and  an  updating  of  imagery  within  a  short  duration 
such  that  visual  perception  is  undisturbed.  The 
critical  duration  within  which  imagery  must  be 
updated  may  be  a  reflection  of  the  time  course  of 
a  kind  of  visual  suppression  called  head  movement 
suppression. 


Head  Movement  Suppression 

Head  movement  suppression  is  a  kind  of  visual 
suppression  that  is  likely  to  be  elicited  during  head 
movements,  especially  voluntary  head  move¬ 
ments.  This  kind  of  visual  suppression  could  serve 
to  make  visual  perception  stable  and  uninterrupt¬ 
ed,  despite  the  rapidly  moving  images  that  sweep 
across  the  retinae  during  a  head  movement,  by 
suppressing  the  perception  of  unwanted  image 
motion.  Head  movement  suppression  would  be 
analogous  to  another  form  of  visual  suppression, 
called  saccadic  suppression.  Because  saccadic 
suppression  has  been  studied  for  many  years, 
many  of  its  properties  are  well  understood.  Knowl¬ 
edge  about  saccadic  suppression  may  inform  re¬ 
search  efforts  on  head  movement  suppression.  For 
this  reason  we  now  briefly  discuss  the  topic  of 
saccadic  suppression. 

Saccadic  suppression  is  the  suppression  of  per¬ 
ception  of  image  displacement  and  image  motion 
during  saccadic  eye  movements,  which  renders 
visual  perception  stable  and  uninterrupted  despite 
the  rapidly  moving  images  that  occur  with  the  eye 
movements.  Such  suppressive  effects  are  typically 
not  noticeable  because  they  are  very  brief,  exist¬ 
ing  about  50  ms  before  the  eye  movement,  extend¬ 
ing  throughout  the  eye  movement,  and  terminating 
about  50  ms  following  the  eye  movement  (Dia¬ 
mond,  Ross,  &  Morrone,  2000).  Saccadic  suppres¬ 
sion  may  be  mediated  at  peripheral  levels  of  visual 
processing  (Roska,  Nemeth,  Orzo,  &  Werblin, 
2000;  Thilo,  Santoro,  Walsh,  &  Blakemore,  2004) 
as  well  as  at  cortical  levels  (Thiele,  Henning, 
Kubischik,  &  Hoffmann,  2002)  and  affect  both 
parvocellular  and  magnocellular  pathways  (Anand 
&  Bridgeman,  2002;  Burr,  Morrone,  &  Ross, 
2002).  Saccadic  suppression  also  may  involve  a 
distortion  of  visual  space  (Cho  &  Lee,  2003)  and 
involve  an  extraretinal  component  (Diamond  et 
al„  2000). 

Turning  now  to  head  movement  suppression, 
it  remains  to  be  determined  whether  the  properties 
of  such  suppression  are  similar  to  those  of  sac¬ 
cadic  suppression  and  what  role  head  movement 
suppression  may  play  in  head  tracking  with  HMDs. 
If  head  movement  suppression  occurs,  and  its 
duration  is  sufficiently  long,  then  the  shift  in  the 
displayed  imagery  on  an  HMD  would  go  un¬ 
noticed  even  with  a  relatively  long  delay.  Howev¬ 
er,  if  the  duration  of  head  movement  suppression 
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is  relatively  short,  then  the  shift  in  displayed 
imagery  would  be  noticed,  which  could  create  the 
conditions  for  unwanted  image  motion  and  per¬ 
ceptual  instability  or  confusion  (Velger,  1998; 
Wells  &  Haas,  1990).  For  the  update  of  imagery  to 
go  unnoticed,  its  delay  would  need  to  be  shorter 
than  the  duration  of  head  movement  suppression. 
Research  should  be  performed  to  investigate  the 
existence  of  head  movement  suppression,  what 
kind  of  information  (e.g.,  movement,  color)  gets 
suppressed,  and  for  how  long.  This  would  allow 
one  to  predict  the  kind  of  perceptual  confusion 
that  might  occur  with  different  head-tracking  lags. 

Head-Tracking  Accuracy 

When  an  HMD  with  conformal  imagery  is 
used,  it  is  important  to  measure  head  position  with 
a  high  degree  of  accuracy  (Martinsen,  Havig,  & 
Post,  2003).  It  is  thought  that  an  accuracy  of  10 
mrad  is  sufficient  for  launching  air-to-air  missiles, 
but  approximately  2  mrad  is  needed  for  weapon 
aiming,  similar  to  HUDs  (Velger,  1998).  Most  sys¬ 
tems  can  meet  an  accuracy  of  about  2  mrad  near 
the  line-of-sight  direction,  but  accuracy  is  degrad¬ 
ed  to  about  10  to  15  mrad  at  large  eccentricities 
from  the  line-of-sight  direction.  Moreover,  the 
head  tracker  should  have  a  sampling  rate  of  at  least 
120  Hz  (Velger,  1998,  p.  177). 

During  head  movements,  users  typically  move 
their  eyes  as  well  as  their  heads,  such  that  the  eye 
movement  leads  the  head  movement;  there  is  an 
angular  offset  between  eye  orientation  and  head 
orientation  (Robinson,  Koth,  &  Ringenbach, 
1976;  Rolland,  Ha,  &  Fidopiastis,  2004;  Velger, 
1998,  p.  173).  Thus,  more  accurate  estimates  of 
line  of  sight  are  available  when  both  eye  move¬ 
ments  and  head  movements  are  measured  (Velger, 
1998,  p.  173).  However,  it  is  difficult  to  measure 
eye  movements  accurately  in  the  context  of  HMDs 
because  many  eye  trackers  are  inaccurate,  heavy, 
and  bulky  (Velger,  1998,  pp.  174-175),  and  in 
many  cases  it  may  be  difficult  to  mount  inside  the 
helmet  the  sensor  needed  to  measure  the  eye 
movements. 

Recommendations 

A  form  of  visual  suppression  called  head  move¬ 
ment  suppression  may  exist  and  may  set  an  upper 
limit  on  the  acceptable  lag  or  delay  for  display  up¬ 
date  when  head-tracking  systems  are  used.  This 
suppression  may  be  exploited  to  possibly  control 


for  the  effects  of  a  perceptual  mismatch  between 
vision  and  proprioception  when  head  movements 
are  performed  with  an  HMD.  The  critical  delay 
for  display  update  is  currently  not  known,  but  it 
may  fall  within  the  range  of  16  to  80  ms.  More 
research  is  needed  to  investigate  the  critical  delay 
for  display  update.  Head-tracking  accuracy  should 
be  about  2  to  10  mrad,  and  the  head  tracker  should 
sample  at  a  rate  of  120  Hz  or  higher. 

SUMMARY 

Based  on  our  review  of  the  literature,  we  make 
the  following  recommendations  for  the  design  and 
use  of  HMDs: 

1 .  Perceptual  constancy  involves  a  cue  integra¬ 
tion  process,  which  could  affect  the  perceived  size, 
depth,  and/or  distance  within  a  scene  presented  on 
the  HMD  if  any  of  the  relevant  cues  are  misreg- 
istered  by  the  visual  system.  Thus,  HMD  bright¬ 
ness  and  contrast  should  be  sufficient  so  as  to 
minimize  the  tendency  for  accommodation  to  drift 
toward  dark  focus  and  vergence  to  drift  toward 
dark  vergence  and,  at  the  same  time,  produce  a  rel¬ 
atively  large  depth  of  focus.  Representative  min¬ 
imum  values  would  be  a  luminance  level  of  at  least 
6.9  cd/m2  and  a  Michelson  contrast  of  at  least  .10. 
The  tendency  for  accommodation  to  drift  toward 
dark  focus,  and  for  vergence  to  drift  toward  dark 
vergence,  may  cause  the  size,  depth,  or  speed  of 
HMD  symbology  to  be  misperceived. 

2.  An  accommodation-vergence  mismatch  can 
create  several  problems  for  the  HMD  user  - 
namely,  eyestrain  or  visual  discomfort,  blurred 
symbology,  and  misperception  of  distance,  size, 
depth,  or  speed  of  objects  presented  on  the  HMD. 
The  remedy  for  an  accommodation-vergence 
mismatch  is  to  maintain  a  steady  convergence 
angle.  More  research  is  needed  to  determine  the 
exact  limits  of  tolerance  for  accommodation- 
vergence  mismatches. 

3.  The  functioning  of  the  parvocellular  and 
magnocellular  pathways  and  the  ventral  and  dor¬ 
sal  cortical  streams  determines,  in  part,  the  type  of 
visual-motor  tasks  that  an  individual  can  perform, 
and  knowledge  of  these  pathways  may  enable 
one  to  better  predict  the  type  of  task  that  may  be 
impaired  with  a  restricted  held  of  view.  Thus,  for 
successful  performance  of  activities  such  as  tar¬ 
geting  and  object  recognition,  a  held  of  view  as 
small  as  40°  may  be  sufficient.  For  successful 
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performance  of  activities  such  as  visual  orienta¬ 
tion  to  peripheral  stimulation,  controlling  the  posi¬ 
tion  of  a  moving  vehicle  relative  to  the  position 
of  other  vehicles  (e.g.,  formation  flying),  or  per¬ 
ceiving  a  full  sense  of  immersion,  a  field  of  view 
much  greater  than  60°  would  likely  be  needed. 
Achieving  both  a  wide  field  of  view  and  high  res¬ 
olution  is  difficult;  relative  design  priorities  for  a 
given  HMD  application  need  to  be  established. 

4.  The  presence  of  mismatched  binocular  input 
may  cause  visual  discomfort  and  disrupt  viewing 
of  an  HMD.  Such  disruption  can  be  minimized  by 
creating  conditions  that  promote  binocular  fusion. 
With  semitransparent  monocular  HMDs,  research 
should  determine  whether  binocular  fusion  of  the 
background  scene  that  is  seen  by  the  two  eyes 
mitigates  against  the  rivalry  that  could  occur  be¬ 
tween  the  imagery  in  the  eye  that  sees  the  HMD 
and  the  background  scene  viewed  by  the  other 
eye.  This  is  important  because  binocular  rivalry 
may  involve  suppression  at  different  levels  of  the 
visual  system.  For  biocular  and  binocular  HMDs, 
representative  tolerance  levels  are  as  follows:  a 
difference  in  luminance  of  no  more  than  30%, 
a  horizontal  difference  of  no  more  than  10  to  23 
arcmin,  a  vertical  difference  of  no  more  than  4  to 
11.5  arcmin,  a  rotational  difference  of  no  more 
than  6  to  12  arcmin,  horizontal  or  vertical  differ¬ 
ences  in  image  size  of  no  more  than  1.5%,  and  a 
deviation  between  centers  of  the  two  displays  of 
no  more  than  0.18  prism  diopters. 

5.  The  possibility  of  the  existence  of  head 
movement  suppression  may  be  exploited  to  con¬ 
trol  for  the  effects  of  a  perceptual  mismatch 
between  vision  and  proprioception  when  head 
movements  are  performed  with  an  HMD.  This  is 
important  because  temporal  delays  or  lags  be¬ 
tween  head  movement  and  update  of  displayed 
imagery  can  impair  human  performance  and  cre¬ 
ate  disorientation,  nausea,  and  discomfort.  The 
critical  value  of  temporal  delay  beyond  which 
vision  becomes  impaired  is  not  consistent  across 
the  literature,  ranging  from  16  to  80  ms.  Research 
should  be  performed  to  determine  the  critical 
delay  and  the  role  that  head  movement  suppres¬ 
sion  may  play  in  the  use  of  HMDs. 

In  summary,  new  insights  for  potential  design 
solutions  for  HMDs,  such  as  those  discussed  in 
the  present  paper,  may  be  gained  by  a  thorough  un¬ 
derstanding  of  basic  visual  functioning.  This  is 
true  even  when  a  solution  for  a  given  problem 


is  not  immediately  apparent;  knowledge  about 
basic  visual  functioning  always  has  the  potential 
of  informing  the  development  of  a  solution  at  a 
later  point  in  time. 
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